Generalist models, which are capable of performing diverse multi-modal tasks in a task-agnostic way within a single model, have been explored recently. Being, hopefully, an alternative to approaching general-purpose AI, existing generalist models are still at an early stage, where modality and task coverage is limited. To empower multi-modal task-scaling and speed up this line of research, we release a generalist model learning system, OFASys, built on top of a declarative task interface named multi-modal instruction. At the core of OFASys is the idea of decoupling multi-modal task representations from the underlying model implementations. In OFASys, a task involving multiple modalities can be defined declaratively even with just a single line of code. The system automatically generates task plans from such instructions for training and inference. It also facilitates multi-task training for diverse multi-modal workloads. As a starting point, we provide presets of 7 different modalities and 23 highly-diverse example tasks in OFASys, with which we also develop a first-in-kind, single model, OFA+, that can handle text, image, speech, video, and motion data. The single OFA+ model achieves 95% performance in average with only 16% parameters of 15 task-finetuned models, showcasing the performance reliability of multi-modal task-scaling provided by OFASys. Available at https://github.com/OFA-Sys/OFASys

图像的美学评估可以分为两种主要形式：数值评估和语言评估。照片的美学标题是已解决的审美语言评估的唯一任务。在本文中，我们提出了一项美学评估的新任务：图像的美学视觉和回答（AVQA）。如果我们提出图像美学问题，模型可以预测答案。我们使用\ textit {www.flickr.com}的图像。目标QA对由提出的美学属性分析算法产生。此外，我们引入了主观质量检查对，这些对从审美数字标签和来自大规模培训模型的情感分析转换。我们构建了第一个回答数据集AESVQA的审美视觉问题，其中包含72,168个高质量图像和324,756对美学问题。已经提出并证明了两种调整数据分布的方法，以提高现有模型的准确性。这是解决美学VQA任务并将主观性引入VQA任务的第一项工作。实验结果表明，我们的方法在这项新任务上的表现优于其他VQA模型。

图像美学质量评估在过去十年中很受欢迎。除数值评估外，还提出了自然语言评估（美学字幕）来描述图像的一般美学印象。在本文中，我们提出了美学属性评估，即审美属性字幕，即评估诸如组成，照明使用和颜色布置之类的美学属性。标记美学属性的注释是一项非平凡的任务，该评论限制了相应数据集的规模。我们以半自动方式构建了一个名为DPC-CAPTIONSV2的新型数据集。知识从带有完整注释的小型数据集转移到摄影网站的大规模专业评论。 DPC-CAPTIONSV2的图像包含最多4个美学属性的注释：组成，照明，颜色和主题。然后，我们根据BUTD模型和VLPSA模型提出了一种新版本的美学多属性网络（AMANV2）。 AMANV2融合了带有完整注释的小规模PCCD数据集和带有完整注释的大规模DPCCAPTIONSV2数据集的混合物的功能。 DPCCAPTIONSV2的实验结果表明，我们的方法可以预测对4种美学属性的评论，这些评论比上一个Aman模型所产生的方法更接近美学主题。通过图像字幕的评估标准，专门设计的AMANV2模型对CNN-LSTM模型和AMAN模型更好。

随着社交软件和多媒体技术的持续发展，图像已成为传播信息和社交的重要载体。如何全面评估图像已成为最近研究的重点。传统的图像美学评估方法通常采用单个数值总体评估评分，该评估具有一定的主观性，无法再满足更高的美学要求。在本文中，我们构建了一个称为Aesthetic混合数据集的新图像属性数据集，该数据集具有属性（AMD-A）和设计融合的外部属性功能。此外，我们还提出了一种有效的方法，用于在混合多属性数据集上进行图像美学属性评估，并通过使用ExtisticNet-B0作为骨干网络来构建多任务网络体系结构。我们的模型可以实现美学分类，整体评分和属性评分。在每个子网络中，我们通过ECA通道注意模块改进特征提取。至于最终的整体评分，我们采用了教师学习网络的想法，并使用分类子网络来指导美学的整体细粒回归。实验结果，使用思维螺旋式的结果表明，我们提出的方法可以有效地改善美学整体和属性评估的性能。

由于其简单性和实用性，密度峰值聚类已成为聚类算法的NOVA。但是，这是一个主要的缺点：由于其高计算复杂性，这是耗时的。在此，开发了稀疏搜索和K-D树的密度峰聚类算法来解决此问题。首先，通过使用k-d树来替换原始的全等级距离矩阵来计算稀疏距离矩阵，以加速局部密度的计算。其次，提出了一种稀疏的搜索策略，以加快与$ k $最近邻居的集合与由数据点组成的集合之间的相互分离的计算。此外，采用了决策值的二阶差异方法来自适应确定群集中心。最后，通过与其他六种最先进的聚类算法进行比较，在具有不同分布特性的数据集上进行实验。事实证明，该算法可以有效地将原始DPC的计算复杂性从$ O（n^2k）$降低到$ O（n（n^{1-1/k}+k））$。特别是对于较大的数据集，效率更加明显地提高。此外，聚类精度也在一定程度上提高了。因此，可以得出结论，新提出的算法的总体性能非常好。

在图像美学质量评估的任务中，由于美学数据集的正常分布，难以达到高分区域和低得分面积。为了减少标签中的错误并解决正常数据分布的问题，我们提出了一个具有名为AMD-CR的分类和回归的新的美学混合数据集，我们培训了元重传网络以重新重量培训数据的损失不同。此外，我们还提供了一种基于二进制分类任务的伪标签的不同阶段的培训策略，然后我们将其用于审美培训，该课程涉及分类和回归任务的不同阶段。在网络结构的构造中，我们构建一种可以适应输入图像的任何大小的美学自适应块（AAB）结构。此外，我们还使用高效的通道注意力（ECA）来加强每个任务的特征提取能力。实验结果表明，与SROCC中的常规方法相比，我们的方法改善了0.1112。该方法还可以帮助找到无人驾驶飞行器（UAV）和车辆的最佳审美路径规划。

The development of social media user stance detection and bot detection methods rely heavily on large-scale and high-quality benchmarks. However, in addition to low annotation quality, existing benchmarks generally have incomplete user relationships, suppressing graph-based account detection research. To address these issues, we propose a Multi-Relational Graph-Based Twitter Account Detection Benchmark (MGTAB), the first standardized graph-based benchmark for account detection. To our knowledge, MGTAB was built based on the largest original data in the field, with over 1.55 million users and 130 million tweets. MGTAB contains 10,199 expert-annotated users and 7 types of relationships, ensuring high-quality annotation and diversified relations. In MGTAB, we extracted the 20 user property features with the greatest information gain and user tweet features as the user features. In addition, we performed a thorough evaluation of MGTAB and other public datasets. Our experiments found that graph-based approaches are generally more effective than feature-based approaches and perform better when introducing multiple relations. By analyzing experiment results, we identify effective approaches for account detection and provide potential future research directions in this field. Our benchmark and standardized evaluation procedures are freely available at: https://github.com/GraphDetec/MGTAB.

A recent study has shown a phenomenon called neural collapse in that the within-class means of features and the classifier weight vectors converge to the vertices of a simplex equiangular tight frame at the terminal phase of training for classification. In this paper, we explore the corresponding structures of the last-layer feature centers and classifiers in semantic segmentation. Based on our empirical and theoretical analysis, we point out that semantic segmentation naturally brings contextual correlation and imbalanced distribution among classes, which breaks the equiangular and maximally separated structure of neural collapse for both feature centers and classifiers. However, such a symmetric structure is beneficial to discrimination for the minor classes. To preserve these advantages, we introduce a regularizer on feature centers to encourage the network to learn features closer to the appealing structure in imbalanced semantic segmentation. Experimental results show that our method can bring significant improvements on both 2D and 3D semantic segmentation benchmarks. Moreover, our method ranks 1st and sets a new record (+6.8% mIoU) on the ScanNet200 test leaderboard. Code will be available at https://github.com/dvlab-research/Imbalanced-Learning.

When using LiDAR semantic segmentation models for safety-critical applications such as autonomous driving, it is essential to understand and improve their robustness with respect to a large range of LiDAR corruptions. In this paper, we aim to comprehensively analyze the robustness of LiDAR semantic segmentation models under various corruptions. To rigorously evaluate the robustness and generalizability of current approaches, we propose a new benchmark called SemanticKITTI-C, which features 16 out-of-domain LiDAR corruptions in three groups, namely adverse weather, measurement noise and cross-device discrepancy. Then, we systematically investigate 11 LiDAR semantic segmentation models, especially spanning different input representations (e.g., point clouds, voxels, projected images, and etc.), network architectures and training schemes. Through this study, we obtain two insights: 1) We find out that the input representation plays a crucial role in robustness. Specifically, under specific corruptions, different representations perform variously. 2) Although state-of-the-art methods on LiDAR semantic segmentation achieve promising results on clean data, they are less robust when dealing with noisy data. Finally, based on the above observations, we design a robust LiDAR segmentation model (RLSeg) which greatly boosts the robustness with simple but effective modifications. It is promising that our benchmark, comprehensive analysis, and observations can boost future research in robust LiDAR semantic segmentation for safety-critical applications.

For Prognostics and Health Management (PHM) of Lithium-ion (Li-ion) batteries, many models have been established to characterize their degradation process. The existing empirical or physical models can reveal important information regarding the degradation dynamics. However, there is no general and flexible methods to fuse the information represented by those models. Physics-Informed Neural Network (PINN) is an efficient tool to fuse empirical or physical dynamic models with data-driven models. To take full advantage of various information sources, we propose a model fusion scheme based on PINN. It is implemented by developing a semi-empirical semi-physical Partial Differential Equation (PDE) to model the degradation dynamics of Li-ion-batteries. When there is little prior knowledge about the dynamics, we leverage the data-driven Deep Hidden Physics Model (DeepHPM) to discover the underlying governing dynamic models. The uncovered dynamics information is then fused with that mined by the surrogate neural network in the PINN framework. Moreover, an uncertainty-based adaptive weighting method is employed to balance the multiple learning tasks when training the PINN. The proposed methods are verified on a public dataset of Li-ion Phosphate (LFP)/graphite batteries.